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BACKGROUND OF THE INVENTION 



1. 



FIELD OF THE INVENTION 



5 



The present invention relates to the field of video compression, and in particular 



to a method and apparatus for distributed intraframe transmission. 

Sun, Sun Microsystems, the Sun logo, Solaris and all Java-based trademarks and 
logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United 
10 States and other countries. All SPARC trademarks are used under license and are 
trademarks of SPARC International, Inc. in the United States and other countries. 
Products bearing SPARC trademarks are based upon an architecture developed by Sun 
Microsystems, Inc. 

15 2. BACKGROUND ART 

Digital video sequences contain large amounts of data. Digital video sequences 
are typically compressed to increase transmission speed and reduce storage space 
requirements. One compression technique, known as interframe compression, encodes 
20 differences between transmitted frames. Since the difference between temporally close 
frames is small, less data is required to transmit the differences between two similar 
frames than to transmit an entire frame. However, as frames become more temporally 
distant from a reference frame, the differences typically increase. Thus, interframe 
compression becomes less effective. To reduce the difference between a reference frame 
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and a transmitted frame, a new, complete reference frame, termed an "I- frame," is 
transmitted periodically. 

Some applications transmit the difference between the transmitted frame and the 
5 previous frame. However, errors sometimes accumulate using this technique. In some of 
these applications, an I-frame is periodically transmitted to compensate for these errors. 

I-frames are images encoded without interframe compression (i.e., without 
reference to any other frame). Thus, it typically requires significantly more data to 
10 represent an I-frame than to represent a non-I-frame. In some applications, transmission 
of I-frames sometimes results in noticeable delays in the digital video transmission. This 
problem can be better understood by reviewing digital video compression. 



Typically, digital video compression algorithms take advantage of both spatial and 
temporal coherence of video streams by using intraframe and interframe compression, 
respectively. Intraframe compression occurs when a frame of a video stream is 
compressed in a manner which does not rely on values in any other frame of the video 
20 stream. Reducing the precision of pixel color representation is one example of intraframe 
compression. Interframe compression occurs when a frame of a video stream is 
compressed in a manner which relies on values in at least one other frame of the video 
stream. Encoding a frame with a set of differences from a previous reference frame is an 
example of interframe compression. 



Digital Video Compression 



15 



25 
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In algorithms such as MPEG-2, interframe compression is achieved by encoding 
the difference between that frame and a previous frame, termed an "intra- frame" or "I- 
frame." This compression method typically results in better compression than encoding 
each frame without reference to other frames. Frequently, frames which are transmitted 
5 in close temporal proximity to one another have fewer differences than frames which are 
not in close temporal proximity to one another. Thus, as an I- frame becomes less recent, 
the compression rate typically falls because the frames being encoded are less similar to 
the I- frame than previous frames. Additionally, transmission errors sometimes result in 
errors in the I-frame and subsequent frames. 

10 

To compensate for the above problems, new I-frames are transmitted periodically. 
The I-frames are typically encoded using intraframe compression. However, no 
interframe compression is used when transmitting I-frames. Thus, representation of I- 
frames requires significantly more data. This increased size slows transmission rates. In 
15 some real-time encoding and/or decoding applications, the large size of I-frames can 

result in noticeable delays or in some frames being dropped. A dropped frame is a frame 
which is either not transmitted or not displayed. 
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SUMMARY OF THE INVENTION 

The present invention provides a method and apparatus for distributed intraframe 
transmission. In one embodiment of the present invention, digital images are divided into 
5 regions. In this embodiment, an I-frame is not transmitted all at once. Instead, for each 
frame, a region of the image is compressed without using interframe compression while 
the rest of the image is compressed using interframe compression. The region transmitted 
without interframe compression changes with each frame. Thus, over the course of 
several frames every region of the image is transmitted without interframe compression, 
yi 10 A transmission schedule controls the transmission of I-frame data for an entire image 
%j over the course of several frame transmissions. 

^ In one embodiment, a digital image is divided into M regions, and in each frame 

JL one region is transmitted without using interframe compression. In one embodiment, the 

^15 M regions are numbered from 0 to M - 1 , and a region is selected for transmission 
N without using interframe compression when the region number is equal to MOD (F, M), 

H where F is the frame number. Thus, each portion of a digital image is guaranteed to be 

refreshed every M frames. The combined size of the one region transmitted without 

interframe compression and the M - 1 regions transmitted using interframe compression 
20 is typically smaller than the size of a complete I-frame. This distributes the computation 

and transmission burdens more evenly over all of the frames and helps to prevent uneven 

delays or dropped frames. 

In one embodiment, the regions are non-overlapping rectangular regions. In 
25 another embodiment, the regions are strips of pixels. In another embodiment, regions are 
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non-contiguous pixels. In yet another embodiment, regions which will be transmitted 
without interframe compression comprise sets of pixels which have not been refreshed 
within M frames and pixels which are selected by a compressibility criteria. In one 
embodiment, the compressibility criteria is that a pixel is included in the region whenever 
5 it is more efficient to not use interframe compression on that pixel. In one embodiment, 
the transmission of the pixels which have not been refreshed within M frames is 
scheduled over multiple frame transmissions. In another embodiment, pixels which are 
determined to be sufficiently unchanged from the last time they were included in a region 
transmitted without using interframe compression are transmitted using interframe 
10 compression. 

In one embodiment, interframe compression is accomplished by making reference 
to the previous frame rather than an I-frame. In this embodiment, a digital image is 
divided into M regions, and a transmission schedule determines when to transmit a region 

15 without using interframe compression. The region transmitted without interframe 
transmission changes with each transmission, so the transmission of I-frame data is 
accomplished over the course of several frame transmissions. Since each transmitted 
frame is typically similar to the immediately preceding frame, interframe compression is 
typically efficient. Thus, the number of frames between regional refreshes allowable 

20 without unacceptable loss of picture quality is typically higher than in the above methods. 
As a result, the size of the interframe-only regions is reduced without unacceptable loss of 
quality, which reduces the amount of I-frame data transmitted with each frame. 

In another embodiment, a video image is divided into M regions and each region 
25 is encoded as an independent movie. In this embodiment, a scheduler is used to insure 
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the independent movies transmit their I-frames at different times. In one embodiment, the 
independent movies are encoded using the MPEG video codec. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features, aspects and advantages of the present invention will 
become better understood with regard to the following description, appended claims and 
5 accompanying drawings where: 

Figure 1 is a flow diagram of the process of distributed I-frame transmission in 
accordance with one embodiment of the present invention. 

10 Figure 2 is a flow diagram of the process of encoding digital video data in 

accordance with one embodiment of the present invention. 

Figure 3 is a block diagram of digital image division in accordance with one 
embodiment of the present invention.. 

15 

Figure 4 is a block diagram of digital image division in accordance with one 
embodiment of the present invention. 

Figure 5 is a block diagram of digital image division in accordance with one 
20 embodiment of the present invention. 

Figure 6 is a flow diagram of the process of encoding digital video data in 
accordance with one embodiment of the present invention. 
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Figure 7 is a flow diagram of the process of determining regions to be transmitted 
without using interframe compression in accordance with one embodiment of the present 
invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

The invention is a method and apparatus for distributed intraframe transmission. 
In the following description, numerous specific details are set forth to provide a more 
5 thorough description of embodiments of the invention. It is apparent, however, to one 
skilled in the art, that the invention may be practiced without these specific details. In 
other instances, well known features have not been described in detail so as not to 
obscure the invention. 

10 In one embodiment of the present invention, digital images are divided into 

regions. In this embodiment, an I-frame is not transmitted all at once. Instead, in each 
frame, a region of the image is compressed without using interframe compression while 
the rest of the image is compressed using interframe compression. The region transmitted 
without interframe compression changes with each frame transmission. Thus, over the 

1 5 course of several frame transmissions every region of the image is transmitted once 

without interframe compression. A transmission schedule controls the transmission of I- 
frame data for an entire image over the course of several frame transmissions. 

Figure 1 illustrates the process of distributed I-frame transmission in accordance 
20 with one embodiment of the present invention. At step 100, a digital image is divided 
into regions. At step 110, one region is selected for intraframe-only (i.e., no interframe) 
compression. At step 120, all regions are transmitted in an order specified by a codec. 
The selected region is transmitted without using interframe compression. At step 130, it 
is determined whether any frames remain to be transmitted. If an untransmitted frame 
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remains, the process repeats at step 110. If no untransmitted frames remain, at step 140, 
the I- frame data transmission is complete. 

In one embodiment, a digital image is divided into M regions, and a region is 
5 transmitted without interframe compression with each frame, hi one embodiment, the M 
regions are numbered from 0 to M - 1, and a region is selected for transmission without 
interframe compression when the region number is equal to MOD (F, M), where F is the 
frame number. Thus, each region of an I- frame is guaranteed to be refreshed every M 
frames. 

10 

Figure 2 illustrates the process of encoding digital video data in accordance with 
one embodiment of the present invention. At step 200, a digital image is divided into M 
regions. At step 205, the regions are numbered consecutively from 0 to M - 1 based on 
the order they are transmitted in the codec. At step 210, F is set to equal to 0. At step 
15 215, a region counter, m, is set equal to 0. At step 220, it is determined whether MOD (F, 
M) = m. If MOD (F, M) = m, at step 225, region m is encoded without using interframe 
compression and the process continues at step 235. If MOD (F, M) is not equal to m, at 
step 230, region m is encoded using interframe compression and the process continues at 
step 235. 

20 

At step 235, m is incremented by 1. At step 240, it is determined whether m is 
less than M. If m is less than M, the process repeats at step 220. If m is not less than M, 
at step 245, the frame is complete and F is incremented by 1. At step 250, it is 
determined whether there are more frames to encode. If there are more frames to encode, 
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the process repeats at step 215. If there are no more frames to encode, at step 255, 
encoding is complete. 

In one embodiment, the sub-groups are non-overlapping rectangular regions. 
5 Figure 3 illustrates an example of digital image division in accordance with one 

embodiment of the present invention. The digital image (300) is divided into sub-groups 
0 (310), 1 (320), 2 (330) and 3 (340). In one embodiment, each sub-group is an n by n 
square of pixels. 

10 In another embodiment, the regions are strips of pixels running the width of the 

image. Figure 4 illustrates an example of digital image division in accordance with one 
embodiment of the present invention. The digital image (400) is divided into regions 0 
(410), 1 (420), 2 (430) and 3 (440). Each region is a strip of pixels. 

15 In another embodiment, regions are non-contiguous regions of pixels. Figure 5 

illustrates an example of digital image division in accordance with one embodiment of 
the present invention. The digital image (500) is divided into pixel groups 0 (501), 1 
(502), 2 (503), 3 (504), 4 (505), 5 (506), 6 (507), 7 (508), 8 (509), 9 (510), 10 (511), 11 
(512), 12 (513), 13 (514), 14 (515), 15 (516), 16 (517), 17 (518), 18 (519), 19 (520), 20 

20 (521), 21 (522), 22 (523) and 23 (524). In this embodiment, pixels are divided into G N 
pixel by N pixel areas of the image. In Figure 5, G is equal to 24. 

These pixel groups labeled from 0 to G - 1 based on the order they are transmitted 
in the codec and are formed into regions. Pixel groups are assigned to regions wherein 
25 MOD (g, M) is equal to the same number for all the pixel groups in a region wherein g is 
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a pixel group's number. In Figure 5, M is equal to 4. Thus, pixel groups 0, 4, 8, 12, 16, 
and 20 comprise region 1. Likewise, pixel groups 1, 5, 9, 13, 17, and 21 comprise region 

2. Additionally, pixel groups 2, 6, 10, 14, 18, and 22 comprise region 3, and pixel groups 

3, 7, 1 1, 15, 19, and 23 comprise region 4. In one embodiment, a pixel group is selected 
5 for transmission without interframe compression whenever MOD (F - g, M) = 0, wherein 

F is the frame number. Thus, if F = 3, the pixel groups of region 4 are encoded without 
using interframe compression. 

Figure 6 illustrates the process of encoding digital video data in accordance with 
10 one embodiment of the present invention. At step 600, F is set equal to 0. At step 605, g 
uj is set equal to 0. At step 610, it is determined whether MOD(|g-F|, M) is equal to 0. If 

^ MOD(|g-F|, M) is equal to 0, at step 615, group g is encoded without interframe 

;HL; compression and the process continues at step 625. If MOD(|g-F|, M) is not equal to 0, at 

^ step 620, group g is encoded with interframe compression and the process continues at 

PU 15 step 625. 

y* At step 625, g is incremented by 1 . At step 630, it is determined whether g is less 

than G. If g is less than G, the process repeats at step 610. If g is not less than G, at step 
635, F is incremented by 1. At step 640, it is determined whether there are more frames 
20 to transmit. If there are more frames to transmit, the process repeats at step 605. If there 
are no more frames to transmit, at step 645, the encoding is complete. 

In yet another embodiment, sub-groups comprise sets of pixels which are selected 
by a compressibility criteria and pixels which have not been refreshed within X frames. 
25 Figure 7 illustrates the process of determining regions to be transmitted without using 
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interframe compression in accordance with one embodiment of the present invention. At 
step 700 m is set equal to 0. At step 710, it is determined whether region m transmitted 
using only intraframe compression within the last X frames. If region m transmitted 
using only intraframe compression within the last frames, at step 720, region m is 
5 compressed twice, using intraframe and interframe compression. At step 730, region m is 
transmitted using the smaller of the two compressions and the process continues at step 
740. 

If at step 710 it is determined that region m was not transmitted using only 
10 intraframe compression within the last X frames, at step 730, region m is transmitted 

using intraframe compression only and the process continues at step 740. At step 740, m 
is incremented by 1. At step 750, it is determined whether m is less than M. If m is less 
than M, the process continues at step 720. It m is not less than M, at step 760, the current 
frame is complete. 

15 

In this embodiment, it is not required to transmit the region transmitted without 
interframe compression all at the same time. A transmission scheduler may send the 
pixel information for the image in the order specified by the codec. The region only 
defines which pixel information is sent without using interframe compression. 

20 

In one embodiment, the transmission of the pixels which have not been refreshed 
within X frames is scheduled over multiple frame transmissions. In another embodiment, 
pixels which are determined to be sufficiently unchanged from the last time they were 
included in a region transmitted without using interframe compression are transmitted 
25 using interframe compression. For example, if the pixels of a region have the same 
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values as the last time the region was compressed without using interframe compression, 
the embodiment determines that the region has not changed significantly. Thus, the 
region is encoded using interframe compression to attain a higher compression rate. 

5 In another embodiment, a video image is divided into M regions and each region 

is encoded as an independent movie. In this embodiment, a scheduler is used to insure 
the independent movies transmit their I-frames at different times. In one embodiment, the 
independent movies are encoded using one of the MPEG video codecs. 

1 0 In one embodiment, interframe compression is accomplished by making reference 

to the previous frame rather than an I- frame. Since each transmitted frame is typically 
similar to the immediately preceding frame, interframe compression is typically efficient. 
In this embodiment, intraframe-only compression is no longer needed to keep the 
reference image current. However, intraframe-only compression is useful for correcting 

15 errors that may accumulate. In this embodiment, a digital image is divided into M 

regions, and a transmission schedule determines when to transmit a region without using 
interframe compression. The region transmitted without interframe compression changes 
with each transmission, so the transmission of I-frame data is accomplished over the 
course of several frame transmissions. Because the only purpose of the intraframe-only 

20 updates is error correction, typically M can be larger than in codecs where intraframe- 
only compression is used to keep the reference image current. Thus, less intraframe-only 
data is transmitted with each frame. 
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Thus, a method and apparatus for distributed intraframe transmission is described 
in conjunction with one or more specific embodiments. The invention is defined by the 
following claims and their full scope and equivalents. 
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